1. Field of the Invention
The present invention relates to an image capturing apparatus that effectively extends a depth of field of an imaging means by performing restoration processing on image data obtained by imaging a subject with the imaging means.
2. Description of the Related Art
An EDoF (extended depth of field) function has been proposed as described, for example, in a literature “Extended depth of field through wave-front coding”, E. R. Dowski, Jr. and W. T. Cathey, Appl. Opt., Vol. 34, No. 11, pp. 1859-1866, 1995. The EDoF function allows acquisition of an image in which a certain range from the focus position is in focus by providing an optical filter for keeping the optical transfer function of the lens optical system constant at a position where the image sensor is disposed over a certain predetermined range of object distances in an optical axis direction so that the optical transfer function of the lens optical system becomes essentially constant within a certain range from the focus position and passing image data obtained by imaging through a restoration filter having a reverse characteristic to a blur characteristic of the imaging optical system. The EDoF function will now be described in detail. FIGS. 4A to 4E illustrate EDoF processing (when the subject is a point image), FIGS. 5A to 5E illustrate EDoF processing (when the subject is an edge image), FIG. 6 illustrates an example characteristic of input light intensity versus output signal value of an image sensor, FIGS. 7A to 7D illustrate EDoF processing with a general image sensor (when the subject is a point image), and FIGS. 8A to 8D illustrate EDoF processing with a general image sensor (when the subject is an edge image).
When a point image like that shown in FIG. 4A is captured by an image capturing apparatus having an EDoF function, the point image is scattered at a position where the image sensor is disposed according to a blur characteristic of the imaging optical system as shown in FIG. 4B. The image sensor of the image capturing apparatus obtains image data of the point image by capturing the scattered point image as shown in FIG. 4C.
Then, the image data are passed through a restoration filter having a reverse characteristic to the blur characteristic of the imaging optical system as shown in FIG. 4D, whereby image data extremely close to the original point image may be obtained as shown in FIG. 4E.
Likewise, when an edge image like that shown in FIG. 5A is captured, the edge image is scattered at a position where the image sensor is disposed according to a blur characteristic of the imaging optical system as shown in FIG. 5B. The image sensor of the image capturing apparatus obtains image data of the edge image by capturing the scattered edge image as shown in FIG. 5C.
Then, the image data are passed through a restoration filter having a reverse characteristic to the blur characteristic of the imaging optical system as shown in FIG. 5D, whereby image data extremely close to the original edge image may be obtained as shown in FIG. 5E.
The above description is based on the assumption that an ideal image sensor having a linear input light intensity versus output signal value characteristic (photoelectric conversion characteristic) is used and therefore the image data (FIGS. 4C and 5C) accurately reflect the blur characteristic of the imaging optical system (FIGS. 4B and 5B).
Recently, however, a logarithmic characteristic and a double linear characteristic having a steep characteristic in a low sensitivity area (as described, for example, Japanese Unexamined Patent Publication No. 2008-017536) have been proposed as the photoelectric conversion characteristic in order to increase the dynamic range of image sensors. Further, it is extremely rare even for general image sensors without having such arrangements to have a perfectly linear characteristic over the entire range as illustrated by the solid line in FIG. 6, although an attempt has been made to realize a substantially linear characteristic over a wide range.
When a point image like that shown in FIG. 4A is captured by an image capturing apparatus having a non-linear photoelectric conversion characteristic as described above, the blur characteristic of the imaging optical system (FIG. 7A) and the signal intensity distribution of image data obtained by the image sensor (FIG. 7B) are different. The restoration filter used for the EDoF processing (FIG. 7C) is designed to have a reverse characteristic to the blur characteristic of the imaging optical system. Therefore, as shown in FIG. 7D, if the blur characteristic of the imaging optical system and the signal intensity distribution of image data obtained by the image sensor are different as described above, an error occurs in which an image signal value representing the point image becomes smaller than the true value (undershoot) even if the image data are passed through the restoration filter, causing a problem that the captured image differs from the original point image.
Likewise, when an edge image like that shown in FIG. 5A is captured, the blur characteristic of the imaging optical system (FIG. 8A) and the signal intensity distribution of image data obtained by the image sensor (FIG. 8B) are different. The restoration filter used for the EDoF processing (FIG. 8C) is designed to have a reverse characteristic to the blur characteristic of the imaging optical system. Therefore, as shown in FIG. 8D, if the blur characteristic of the imaging optical system and the signal intensity distribution of image data obtained by the image sensor are different as described above, an error occurs in which an image signal value becomes larger than the true value (overshoot) in the upper side (left side in FIG. 8D) of the edge image and an image signal value becomes smaller than the true value (undershoot) in the lower side (right side in FIG. 8D) of the edge image even if the image data are passed through the restoration filter, causing a problem that the captured image differs from the original edge image.
It is, of course, not always the case that the error mode will become like that described above due to the difference in blur characteristic of the imaging optical system and photoelectric conversion characteristic of the image sensor. In any case, the use of an image sensor having a non-linear photoelectric conversion characteristic, as described above, will cause an error in the restored image.
Here, the description has been made of a case in which a point image and an edge image are used as the subject, but such phenomenon is not limited to these images and will occur in any type of subject.
In image sensors provided with RGB color filters and capable of obtaining a color image, it is often the case that the non-linear characteristic differs from color to color. Further, in image sensors formed of a plurality of types of image sensors having different photoelectric conversion sensitivity values, it is often the case that the difference in non-linear characteristic occurs between high and low sensitivity image sensors.
The present invention has been developed in view of the circumstances described above, and it is an object of the present invention to provide an image capturing apparatus that effectively extends a depth of field of an imaging means by performing restoration processing on image data obtained by imaging a subject with the imaging means and is capable of obtaining more accurately restored image data even when the imaging is performed using an image sensor having a non-linear photoelectric conversion characteristic.